Generalisation in Cubic Nodes - centres and clustering First, recall the action of TLUs for comparison. The operation of an

This lecture deals with training nets of cubic nodes and introduces another major (quite general) algorithm-Reward Penalty. Insight into how we might train nets of cubic nodes is provided by considering the problems associated with generalisation in these nets. We then go on to consider feedback or recurrent nets from the point of view of their implementing iterated feedforward nets (recall this discussion in the case of Hoppeld nets). Although the discussion here centres on cubic nodes, it provides insight into recurrent nets quite generally. Reward Penalty is introduced and is shown to apply to cubic as well as semilinear nodes. 1 Generalisation in Cubic Nodes-centres and clustering First, recall the action of TLUs for comparison. The operation of an n-input TLU on Boolean vectors is determined by a hyperplane passing through the n-cube, so that all vectors on one side of this plane will produce a `1', while the others generate a `0'. Suppose a TLU has been trained to classify two input vectors. Every other possible input pattern will now be classiied according to the node's hyperplane and there is automatic generalisation across the whole input space. (Recall the training of a 2-input TLU with only 2 vectors). Consider now, a cubic node which, in the untrained state, has all sites set to zero. The output to any vector will be totally random with there being equal probability of a 1 or a 0. If this node is now trained on two (Boolean) vectors, only the two sites addressed by these will have their values altered; any other vector will produce a random output and there has been no generalisation. We shall call sites addressed by the training set centre sites or centres. In order to promote Hamming distance generalisation, sites close to the centres need to be trained to the same or similar value as the centres themselves. That is, there should be a clustering of site values around the centres. This is true in both feedforward and recurrent cube-based nets and there are various ways of achieving this which are discussed later. The situation is shown schematically in the diagram attached.